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Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 
• Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )^ Responsive to communication(s) filed on 25 September 2006 . 
2a)D This action is FINAL. 2b)S This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-28 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) Q Claim(s) is/are allowed. 

6) ^ Claim(s) 1-28 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) Q Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1 .121 (d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) Q Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)Q All b)Q Some * c)Q None of: 

1 .□ Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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Continued Examination Under 37 CFR 1.114 

1 . A request for continued examination under 37 CFR 1.114, including the fee set forth in 
37 CFR 1.17(e), was filed in this application after final rejection. Since this application is 
eligible for continued examination under 37 CFR 1.1 14, and the fee set forth in 37 CFR 1.17(e) 
has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 
37 CFR 1.1 14. Applicant's submission filed on 09/25/2006 has been entered. 

Response to Amendment and Arguments 

2. Applicant's arguments with regard to claims 1-28 have been fully considered, but are not 
considered persuasive because of the following reasons: 

Regarding independent claims 1, and 14, the Applicant argues (pages 1 1-14 of the 
Remarks) that Jain does not disclose the limitation "wherein the perceptual features and their 
combinations are derived at least in part through subjective experiments performed with human 
observers". The Examiner respectfully disagrees, a handful of cited locations column 4, lines 
21-32; FIG. 1A, elements 102, 104 and 112; column 6, lines 58-61; column 8, lines 32-35; 
column 11, lines 43-59 and column 18, lines 7-26 are clearly show that human 
observers/users/operators are heavily involved in the process of driving perceptual features and 
category processing. 

Thus, the rejections of all of the claims are maintained. 

Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 
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(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

4. Claims 1, 3-28 are rejected under 35 U.S.C. 102(b) as being anticipated by Jain et al. U.S. 
Patent No. 5,915,250. 

Regarding claim 1, Jain teaches a computer implemented method (FIG. 1A) for 
determining the semantic meaning of mages, comprising: 

deriving a set of perceptual semantic categories (defining and register custom primitives) 
(abstract) for representing important semantic cues (object's attributes) (abstract) (color, 
texture, shape, pattern, object) (face) (column 4, lines 1-45) in the human perception of images 
(column 4, lines 20-25), where each semantic category is modeled through a combination of 
perceptual features that define the semantics of that category and that discriminate that category 
from other categories (visual information retrieval system provides modular and framework to 
define and classify category of object's attributes) (abstract and column 3, lines 60 to column 4, 
lines 1-10), wherein the perceptual features and their combinations are derived at least in part 
through subjective experiments performed with human observers (column 4, lines 21-32; FIG. 
1A, elements 102, 104 and 112; column 6, lines 58-61; column 8, lines 32-35; column 11, lines 
43-59 and column 18, lines 7-26); and 

for each semantic category, forming a set of the perceptual features as a complete feature 
set CFS (feature vector to store each kind of primitives) (column 5, lines 5-20). 

For claim 3, Jain further teaches a method further comprising extracting perceptual 
features from an input image and applying a perceptually-based metric (a similarity scoring 
system) to determine the semantic category for that image (abstract). 
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Referring to claim 4, Jain discloses a method comprising processing the input image to 
compute the CFS (FIG. 1 A, element 1 12); comparing the input image to each semantic category 
through the perceptually-based metric that computes a similarity measure between the features 
used to describe the semantic category and the corresponding features extracted (abstract) from 
the input image (rank similarity result) (column 12, lines 50-67); and assigning the input image 
to the semantic category that corresponds to a highest value of the similarity measure (column 
12, lines 65 to column 13, line 3). 

For claims 5, Jain also discloses a method further comprising computing features from 
the CFS for images in an image database (FIG. 1 A); and generating a distance measure for 
characterizing a relationship of a selected image to another image in the image database by 
applying a perceptually-based similarity metric (column 8, lines 10-27). 

For claim 6, Jain shows a method where values of the similarity metric computed for 
images in the image database are subsequently used to search for similar images 
in the image database (column 9, lines 50-64). 

Regarding claim 7, Jain shows a method where values of the similarity metric computed 
for images in the image database are subsequently used to organize images in the 
image database (classifying images base on similarity scoring system) (FIG. 5B, elements 280 
and 284). 

For claim 8, Jain teaches a method where values of the similarity metric computed for 
images in the image database are subsequently used to display images in the image database in 
an organized manner (FIG. 4; FIG. 5B, element 294; and column 11, lines 23-39). 
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Also to claim 9, Jain further teaches a method further comprising defining a subset of 
features for the selected image or for an image retrieved from the image database, and using the 
subset of features to refine a search through the image database (searching with specific 
parameters) (column 4, lines 29-45). 

Referring to claim 10, Jain shows a method wherein the image database is located at a 
remote location and is reachable through a data communications network (FIG. IB and column 
9, lines 10-25). 

Also to claim 11, Jain further shows a method wherein the image database is located at a 
remote location and is reachable through a data communications network, and where the step of 
characterizing the relationship of the selected image to another image in the image database by 
applying the perceptually-based similarity metric is accomplished to retrieve an image from the 
remote image database (FIG. IB and column 9, lines 10-50). 

As for claim 12, Jain also teaches a method wherein the image database is located at a 
remote location and is reachable through a data communications network, and where the 
step of characterizing the relationship of the selected image to another image in the image 
database by applying the perceptually-based similarity metric is accomplished in conjunction 
with a text-based search algorithm to retrieve a multi-media object from the remote location 
(direction communication between databases) (column 9, lines 10-50). 

Referring to claim 13, Jain discloses a method wherein to assign a particular semantic 
category to an image all of a set of Required Features must be present in the image, and at least 
one of a set of Frequently Occurring Features must be present in the image (fixed/universal or 
default primitives) (column 8, line 60 and column 16, lines 16-25). 



Application/Control Number: 1 0/033,597 Page 6 

Art Unit: 2624 

For claim 14, please refer back to claims 1-3 for further teachings and explanations. In 
addition, Jain teaches a data processing system comprising a data processor, a graphical user 
interface and a memory to processes the aforementioned limitations (FIG. 1 A and column 9, 
lines 5-50). 

For claims 15-24, please refer back to claims 4-13 respectively for further teachings and 
explanations. 

For claim 25, please refer back to claims 1-4 for further teachings and explanations. In 
addition, Jain teaches a computer program (column 6, lines 15-25) to processes aforementioned 
limitation and semantic categories being modeled using multidimensional scaling and 
hierarchical clustering techniques (different ways of querying and classifying images) (FIG. 1 A, 
elements 106 and 108). 

For claims 26 - 27, Jain teaches a computer program where as a result of comparing the 
input image to images stored in the image database one or more most similar 
images are identified in the image database (column 12, lines 50-67 and (column 12, lines 65 to 
column 13, line 3) and display it (FIG. IB, element 148). 

For claim 28, please refer back to claim 14 for further teachings and explanations. 
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Contact Information 



5. 



Any inquiry concerning this communication or earlier communications from the 



examiner should be directed to Brian Q. Le whose telephone number is 571-272-7424. The 
examiner can normally be reached on 8:30 A.M - 5:30 P.M. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Jingge Wu can be reached on 571-272-7429. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 




Brian Le 
October 26, 2006 



